Skip to content

chore(spec): spec liveness gate — classify-or-fail for authorable properties#1919

Merged
os-zhuang merged 1 commit into
mainfrom
chore/spec-liveness-gate
Jun 15, 2026
Merged

chore(spec): spec liveness gate — classify-or-fail for authorable properties#1919
os-zhuang merged 1 commit into
mainfrom
chore/spec-liveness-gate

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

First step of the long-term "close the spec↔runtime gap" direction. For a metadata-driven platform the spec is the product surface — a parsed-but-unenforced property is a silent no-op, and for a security property a silent no-op is false compliance (e.g. forceMfa: true accepted and ignored). The metadata-liveness audits found large DEAD swaths; this makes the classification explicit and regression-proof.

What it does

Every authorable property in a governed category must declare a runtime-liveness status with evidence in packages/spec/liveness/<category>.json, or CI fails — the ratchet: no new undeclared surface.

Status Meaning
live has a runtime consumer (cite file:line)
experimental / planned declared, intentionally not enforced (also read from [EXPERIMENTAL — not enforced] spec markers)
dead parsed, no consumer → enforce-or-remove worklist
internal runtime DTO/result, not authorable (exempt)

Resolution: ledger entry → spec .describe() marker → UNCLASSIFIED.

Seeded: security (the P0 category)

93 authorable properties — 66 dead, 26 live, 1 experimental. ~71% of the authorable security surface is parsed-but-unenforced. Seeded from docs/audits/2026-06-security-identity-property-liveness.md (file:line evidence) + greps for what the audit didn't cover. The dead set is the concrete worklist for the security enforce-or-remove ADR — most urgently the ungated destructive ObjectPermission.allow{Transfer,Restore,Purge} and the entirely-dead Policy tree (password/session/forceMfa/network/audit).

Pieces

  • packages/spec/scripts/liveness/check-liveness.mjs — the gate (--dump, --json).
  • packages/spec/liveness/security.json — the seeded ledger.
  • .github/workflows/spec-liveness-check.yml — runs on PRs touching packages/spec/**.
  • pnpm --filter @objectstack/spec check:liveness + packages/spec/liveness/README.md.

Verified

  • Gate green on security (0 unclassified, exit 0).
  • Ratchet fires: injecting a new ObjectPermission flag → unclassified → exit 1.
  • json-schema/ is generated (gitignored); CI regenerates via gen:schema before checking.

Rollout

Governed today: security only. Other categories (data, automation, ui, …) are added one at a time, highest-risk-first, each seeded from its existing audit — see the README. Same "drift → CI gate" pattern as the docs-accuracy system (#1906).

🤖 Generated with Claude Code

…perties

For a metadata-driven platform the spec IS the product surface; a parsed-but-
unenforced property is a silent no-op, and for security props a silent no-op is
false compliance (e.g. forceMfa accepted and ignored). The metadata-liveness
audits found large dead swaths. This makes the classification explicit and
regression-proof.

- packages/spec/scripts/liveness/check-liveness.mjs — reads the generated
  json-schema/<category>/*.json, resolves each authorable property's liveness
  (ledger entry > spec .describe() marker > UNCLASSIFIED), and exits non-zero on
  any unclassified property in a GOVERNED category (the ratchet: no new
  undeclared surface). --dump inventories a category; --json for machines.
- packages/spec/liveness/security.json — security ledger seeded from
  docs/audits/2026-06-security-identity-property-liveness.md (file:line evidence)
  plus greps for schemas the audit didn't cover. 93 props: 66 dead, 26 live,
  1 experimental. The dead set (Policy tree, allow{Transfer,Restore,Purge},
  isProfile, contextVariables, SharingRule, Territory, RLSConfig) is the
  enforce-or-remove worklist.
- .github/workflows/spec-liveness-check.yml — runs the gate on PRs touching
  packages/spec/** (gen:schema then check).
- check:liveness npm script + packages/spec/liveness/README.md (how to roll out
  the next category, highest-risk-first).

Governed today: security only. Other categories are added one at a time as their
ledgers are seeded from the existing audits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
spec Ready Ready Preview, Comment Jun 15, 2026 3:18pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec.

89 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/concepts/architecture.mdx (via @objectstack/spec)
  • content/docs/concepts/cloud-artifact-api.mdx (via packages/spec)
  • content/docs/concepts/cluster-semantics.mdx (via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx (via packages/spec)
  • content/docs/concepts/implementation-status.mdx (via @objectstack/spec)
  • content/docs/concepts/index.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx (via packages/spec)
  • content/docs/concepts/north-star.mdx (via packages/spec)
  • content/docs/concepts/packages.mdx (via @objectstack/spec)
  • content/docs/concepts/setup-app.mdx (via @objectstack/spec)
  • content/docs/concepts/skills.mdx (via @objectstack/spec)
  • content/docs/concepts/webhook-delivery.mdx (via @objectstack/spec)
  • content/docs/getting-started/architecture.mdx (via @objectstack/spec)
  • content/docs/getting-started/cli.mdx (via @objectstack/spec)
  • content/docs/getting-started/core-concepts.mdx (via @objectstack/spec)
  • content/docs/getting-started/examples.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx (via @objectstack/spec)
  • content/docs/guides/adding-a-metadata-type.mdx (via @objectstack/spec)
  • content/docs/guides/ai-capabilities.mdx (via @objectstack/spec)
  • content/docs/guides/airtable-dashboard-analysis.mdx (via @objectstack/spec)
  • content/docs/guides/analytics-datasets.mdx (via @objectstack/spec)
  • content/docs/guides/api-reference.mdx (via @objectstack/spec)
  • content/docs/guides/business-logic.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/backward-compatibility.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/error-catalog.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/field-type-gallery.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/field-validation-rules.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/permissions-matrix.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/protocol-diagram.mdx (via packages/spec)
  • content/docs/guides/cheatsheets/query-cheat-sheet.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/quick-reference.mdx (via @objectstack/spec)
  • content/docs/guides/client-sdk.mdx (via @objectstack/spec)
  • content/docs/guides/common-patterns.mdx (via @objectstack/spec)
  • content/docs/guides/contracts/auth-service.mdx (via packages/spec)
  • content/docs/guides/contracts/cache-service.mdx (via packages/spec)
  • content/docs/guides/contracts/data-engine.mdx (via @objectstack/spec)
  • content/docs/guides/contracts/index.mdx (via @objectstack/spec)
  • content/docs/guides/contracts/metadata-service.mdx (via packages/spec)
  • content/docs/guides/contracts/storage-service.mdx (via packages/spec)
  • content/docs/guides/data-modeling.mdx (via @objectstack/spec)
  • content/docs/guides/deployment-vercel.mdx (via @objectstack/spec)
  • content/docs/guides/driver-configuration.mdx (via @objectstack/spec)
  • content/docs/guides/error-handling-client.mdx (via @objectstack/spec)
  • content/docs/guides/error-handling-server.mdx (via @objectstack/spec)
  • content/docs/guides/formula.mdx (via @objectstack/spec)
  • content/docs/guides/hook-bodies.mdx (via packages/spec)
  • content/docs/guides/kernel-services.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/dashboard.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/field.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/flow.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/index.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/object.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/validation.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/workflow.mdx (via @objectstack/spec)
  • content/docs/guides/packages.mdx (via @objectstack/spec)
  • content/docs/guides/plugin-development.mdx (via @objectstack/spec)
  • content/docs/guides/plugins.mdx (via @objectstack/spec)
  • content/docs/guides/project-scoping.mdx (via @objectstack/spec)
  • content/docs/guides/public-forms.mdx (via @objectstack/spec)
  • content/docs/guides/runtime-services/email-service.mdx (via packages/spec)
  • content/docs/guides/runtime-services/index.mdx (via packages/spec)
  • content/docs/guides/runtime-services/queue-service.mdx (via packages/spec)
  • content/docs/guides/runtime-services/sharing-service.mdx (via packages/spec)
  • content/docs/guides/runtime-services/storage-service.mdx (via packages/spec)
  • content/docs/guides/security.mdx (via @objectstack/spec)
  • content/docs/guides/seed-data.mdx (via @objectstack/spec)
  • content/docs/guides/skills.mdx (via @objectstack/spec)
  • content/docs/guides/standards.mdx (via @objectstack/spec)
  • content/docs/guides/troubleshooting.mdx (via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx (via @objectstack/spec)
  • content/docs/protocol/objectos/config-resolution.mdx (via @objectstack/spec)
  • content/docs/protocol/objectos/i18n-standard.mdx (via @objectstack/spec)
  • content/docs/protocol/objectos/lifecycle.mdx (via @objectstack/spec)
  • content/docs/protocol/objectos/plugin-spec.mdx (via @objectstack/spec)
  • content/docs/protocol/objectos/runtime-capabilities.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx (via packages/spec)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx (via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx (via packages/spec)
  • content/docs/protocol/objectui/record-alert.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx (via @objectstack/spec)
  • content/docs/releases/index.mdx (via @objectstack/spec)
  • content/docs/releases/v9.mdx (via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation ci/cd dependencies Pull requests that update a dependency file tooling labels Jun 15, 2026
@os-zhuang
os-zhuang merged commit 43ecc08 into main Jun 15, 2026
16 checks passed
@os-zhuang
os-zhuang deleted the chore/spec-liveness-gate branch June 15, 2026 15:39
os-zhuang added a commit that referenced this pull request Jun 18, 2026
…2024)

* docs(adr): ADR-0054 prove-it-runs gate for the authorable surface

Extends ADR-0049 (enforce-or-remove) with a third leg. The liveness ledger
(#1919) classifies every authorable property live/experimental/dead, but "live"
means only a static file:line consumer pointer — proof that something reads the
property, not that authoring it produces correct runtime behavior. #2018 (tz
bucketing: live at every layer, broken in integration) and the field-type
fidelity gaps (#2022: rating/slider/toggle read back wrong-typed) fell through
that gap — call it "unproven liveness".

For a platform whose authors are AI emitting metadata across a combinatorial
space the examples never cover, unproven liveness ships silently into
third-party apps. ADR-0054 upgrades a `live` classification to optionally carry a
`proof` — a @objectstack/dogfood test that authors the property against the real
in-process stack and asserts the runtime outcome. Required as a ratchet (not a
retrofit) for a high-risk authorable class on change, and for any property
implicated in a shipped regression (the fix carries its proof). Generative
testing is explicitly deferred (Phase 3, evidence-gated).

Proposed — for architect review.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(adr): accept ADR-0054 (prove-it-runs gate)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
os-zhuang added a commit that referenced this pull request Jun 23, 2026
fix(grid): rows-per-page selector honors pagination.pageSizeOptions; drop duplicate ListView <select> (#1919)

objectui@92c32428ff6ff1e488e0c233777db654233afede
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation size/m tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant